The invention will be described, for illustration, in the context of a robotic arc-welding seam tracking guidance of the robot, or other manipulator of an effector end, to follow the seam path as sensed ahead of the tool, and secondly, image processing conceived and implemented for the detection of the seam path and for the generation of parametric information to be used in controlling the industrial robot.
It is generally known to convert a video signal from a 2-D image into a digital signal and to process such digital signal in order to:
(1) improve the quality of the image;
(2) determine the overall geometric characteristics of the image;
(3) derive the coordinates of a characteristic point of the image; and
(4) control the motiion of an effector end along a seam path, as a function of the point coordinates sensed along the seam.
See for instance:
(1) "Joint Tracking and Adaptive Robotic Welding Using Vision Sensing of the Weld Joint Geometry" by J. E. Agapakis, J. D. Katz, M. Koifman, G. N. Epstein, J. M. Friedman, D. O. Eyring and H. J. Rutishauser in Welding Journal, November 1986, pp 33-41.
(2) U.S. Pat. No. 4,616,121 issued Oct. 7, 1986.
It is known to treat a digital array by pipeline processing in order to improve, transform, or dissect the information. See for instance:
(1) "Real-Time Signal Processing With Two-Dimensional Filters" by Richard J. Jones, Joseph R. Burns and David C. Smith in SPIE, Vol. 156, Modern Utilization of Infrared Technology IV (1978), pp 43-48; and
(2) "A Study of Pipelining in Computing Arrays" by Hosagrahar, V. Jagadish, Rob. G. Mathews, Thomas Kailath and John A. Newkird in IEEE Transactions on Computers, Vol. C-35, No. 5, May 1986, pp 431-439.
See also "Digital Image Processing" by Rafael C. Gonzales and Paul Wintz, published by Addison-Wesley Publishing Co. in 1977; and, "Digital Processing," edited by Lawrence R. Rabiner and Charles M. Rader, a reprint by IEEE Press, NY (Audio and Electroacoustics Group).
It is also known in image processing to derive a binary-coded digital representation of each pixel constituting a 2-D video image together with neighboring pixels, and to digitally treat such coded digital representation in order to improve the quality of the image. For such treatment, the image is divided into elementary areas called either a block, a mask, or a matrix of pixels each centered on one pixel, and the overall, or average value, is determined for the center and its neighboring pixels according to a predetermined code, or rule of comparison, the entire picture being scanned through its lines and columns for each pixel thereof. This technique has been used to delineate the contour of an image, i.e., of a white image against its black background. See, for instance, U.S. Pat. Nos. 4,672,463; 4,670,793; 4,638,694; and, 4,638,369.
Rather than looking for the sharpness of the image and delineating the edge thereof, there is a different approach, not considered in the prior art, which consists in reducing the image to a line defined by the edge. This is the approach chosen as the basis for one aspect of the present invention. It is most suited for seam tracking, since an important aspect of this technique resides in identifying the seam to be followed by a robot. The stripe technique used generally to recognize a seam, or joint, and to determine by its coordinates the critical center of the seam being followed by the sensor and the effector end, requires a sharp line for accuracy of control. Therefore, it is one main object of the present invention to reduce the stripe image of a seam derived in seam tracking to a line defined by one edge of the image. While so doing, the invention provides for the conversion of the stripe image into a line one pixel wide.
This technique is in contrast to the prior art existing on edge extraction, image thinning, or line extraction, which are generally described by the following articles:
(1) "On Detecting Edges" by Vishvijt S. Nalwa and Thomas O. Binford in IEEE Transations, Vol. PAMI-8, No. 6, November 1986;
(2) "A Width-Independent Fast Thinning Algorithm" by Carlo Arcelli and Gabriella Sanniti Di Baja in IEEE Transactions, Bol. PAMI-7, No. 4, July 1985; and
(3) "Extracting Straight Lines" by J. Brian, Allen R. Hanson and Edward M. Riseman in IEEE Transactions, Vol. PAMI-8, No. 4, July 1986.
Having obtained an image of the stripe reduced to a one-pixel wide line following the edge, before finding at any location of the stripe along the seam, as sensed, what the seam coordinates actually are, there remains to determine what the general features of the seam are and among those features to recognize where the critical corner, or center, is located which is characterizing both the seam sensing path and the gap to be accounted for by the effector end. For this purpose, operations are performed on the extracted stripe line at a high image processing level which comprises:
(1) point sorting;
(2) stripe modeling;
(3) shape matching; and
(4) 3-D coordinate conversion.
Such steps, applied in another context than a pixel wide light stripe image, may be recognized in the aforementioned Agapakis articlr of the Welding Journal of November 1986 (pp 33-41), and also in U.S. Pat. Nos. 4,616,121 and 4,596,919.
In contrast, the processing steps as performed in accordance with the present invention, because of the nature of the electrical definition of the light stripe image as a one-pixel wide line, impart simplicity and a rapidity of execution as best suited for real time robot control with an optical seam tracker, and in particular with a robotic seam tracker system such as described in the aforementioned copending patent application, Ser. No. 07/139,890, filed Dec. 31, 1987.